Point and Interval Estimation

• Last week, we talked about estimating population proportions.
• Our point estimates were proportions of a random sample, drawn with replacement, from the population.
• To we’d be, we used interval estimates. In particular, calibrated interval estimates. Confidence intervals.
  • To get these intervals, we add ‘arms’ of equal length to our point estimate.
  • We choose the arm length so that, 95% of the time, the population proportion is within them.

Calibration of Interval Estimates

• We can do that using the sampling distribution of our point estimate.
• We’d draw arms out from the sampling distribution’s until they span 95% of point estimates.
• We can see that this is the right length by a shift of perspective.
  • A point estimate can touch the mean with its arms iff¹ the mean can touch it with equally long arms.
  • So 95% of intervals cover if and only if 95% of point estimates are within the mean’s arms.
• In practice, we can’t do exactly that. We don’t know the sampling distribution.
• But we can use an estimate of the sampling distribution in its place.
• If it’s a good one, we’ll get approximately 95% coverage.

A Parametric Estimate of the Sampling Distribution

• It helps to know the parametric form of the sampling distribution: it’s binomial.
  • It’s the distribution of the proportion heads in 625 flips of a coin with probability \(p \approx 0.70\) of heads.
  • where \(p \approx 0.70\) is the proportion of registered voters who will vote.
• By parametric form, I mean a formula in terms of some parameters.
  • That tells us what parameters we need to estimate to estimate the sampling distribution.
  • In this case, the only (unknown) parameter is the probability of heads, \(p\).

A Parametric Estimate of the Sampling Distribution

• In R, dbinom(x, n, p), calculates this formula where …
  • x is the number of heads
  • n is the number of flips
  • p is the probability of heads, i.e., the proportion \(\bar y\) of registered voters who will vote.
• To estimate the sampling distribution, we plug in our point estimate \(\bar Y\) for \(p\).
• There’s another way to think about this that generalizes to more complicated situations.

The Bootstrap Interpretation

• The estimated sampling distribution is \(\text{Binomial}(n,p)/n\) for \(n=625\) and \(p=\bar Y\approx 0.68\).
  • It’s the distribution of the proportion heads in 625 flips of a coin with probability \(p \approx 0.68\) of heads.
  • where \(p \approx 0.68\) is the proportion of voters we’ve polled who will vote.
• That is, it’s the sampling distribution of a ‘poll’ of the people in our sample, i.e.
  • roll a 625-sided die 625 times
  • call up the corresponding person in our sample
  • and counting up the yeses we hear
• Note that this is random because we’re drawing with replacement.
  • Each time we run this poll, we call each person in our sample 0,1,2,… times
  • And the number of times we call them is random.

The Bootstrap Interpretation

• If we plot our voters on a map, you can see the idea in visual terms.
  • On the left, we have the population.
  • In the middle, we have our sample. It’s drawn, with replacement, from the population.
  • On the right, we have something else. A new sample drawn, with replacement, from the sample.
• Each ‘call’ that a person receives increases the size of their dot: \(\text{circle area} \propto \text{number of calls}\).
• In the sample, even though it’s drawn with replacement, all dots are the same size.
  • Because we draw from such a large population, nobody gets called twice.
• In the bootstrap sample, dots vary in size.
  • Because we draw \(n\) people from a sample of size \(n\), it’s almost impossible not to call somebody twice.

Bootstrapping

• Before the election, we don’t observe the population. But we do observe the sample.
  • And we can sample from the sample act as if it were the population.
  • We’ll take repeated random samples of size 625, with replacement, from our sample of size 625.
• We call these bootstrap samples and estimates based on them bootstrap estimates.
  • The distribution of these estimates is called the bootstrap sampling distribution.
  • If the sample is like the population, this should be like our estimator’s actual sampling distribution.

The Bootstrap is Nonparametric

bootstrap.samples = array(dim=10000)
for(rr in 1:10000) {
    Y.star = Y[sample(1:n, n, replace=TRUE)]
    bootstrap.samples[rr] = mean(Y.star) 
}

• We do not need to know the parametric form of our sampling distribution to use it.
• All we do is re-run our poll acting as if our sample were the population.
  • We can do this no matter what we’re estimating.
  • So let’s try it out on some other stuff.

Comparing Black and Non-Black Turnout

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{question} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X}_{625} & \overline{Y}_{625} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 1} & \text{difference} \\ \text{outcome} & \underset{\textcolor{gray}{x_{869369}}}{0} & \underset{\textcolor{gray}{y_{869369}}}{1} & \underset{\textcolor{gray}{x_{4428455}}}{1} & \underset{\textcolor{gray}{y_{4428455}}}{1} & \dots & \underset{\textcolor{gray}{x_{1268868}}}{0} & \underset{\textcolor{gray}{y_{1268868}}}{1} & 0.28 & 0.68 & 0.68 & 0.69 & 0.01 \\ \end{array} } \]

• It’s useful to predict turnout among Black voters because they tend to vote differently than non-Black voters.
• Let’s suppose that we’re interested in the difference in turnout between Black and non-Black voters.
• This is a bit reductive, but it’s the beginning of the semester, so we’re keeping things simple.
• We know from the voter file that 30% of registered voters are Black.
• Let’s suppose we asked the people we polled if they were Black, recording their answer in a covariate \(X_i\).

Comparing Black and Non-Black Turnout

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{question} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X}_{625} & \overline{Y}_{625} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 1} & \text{difference} \\ \text{outcome} & \underset{\textcolor{gray}{x_{869369}}}{0} & \underset{\textcolor{gray}{y_{869369}}}{1} & \underset{\textcolor{gray}{x_{4428455}}}{1} & \underset{\textcolor{gray}{y_{4428455}}}{1} & \dots & \underset{\textcolor{gray}{x_{1268868}}}{0} & \underset{\textcolor{gray}{y_{1268868}}}{1} & 0.28 & 0.68 & 0.68 & 0.69 & 0.01 \\ \end{array} } \]

• In our poll, we found that…
  • 172 of the people we called (28%) were Black with a turnout rate of 0.69
  • 453 of the people we called (72%) were non-Black with a turnout rate of 0.68
    
• After the election, we found that …
  • Turnout among Black registered voters was 0.74
  • Turnout among non-Black registered voters was 0.68.
• We’d estimate the difference to be \(0.69-0.68 \approx 0.01\).
• The actual difference was \(0.74-0.68 \approx 0.06\).
• Depending on what you’re doing, that point estimate might be good enough or might not be.
• It’d be nice to have an interval estimate.

A Table of Imaginary Polls

$$

The Sampling Distribution

difference.samples = array(dim=10000)
for(rr in 1:10000) {
    I = sample(1:m, n, replace=TRUE)
    X = x[I]
    Y = y[I]
    difference.samples[rr] = mean(Y[X==1]) - mean(Y[X==0])
}

• Here’s what the sampling distribution of this difference in turnout looks like.
• As before, if we can estimate it we can use that estimate to get a 95% confidence interval.
• But, unlike before, we don’t really know the parametric form of its sampling distribution.
• Or—at least—it’d be a pain to work it out. So we’ll use the bootstrap to estimate it.

A Table of Bootstrap Polls

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{poll} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X} & \overline{Y} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 1} & \text{difference} \\ \hline \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}\dots & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0.28 & \color[RGB]{7,59,76}0.68 & \color[RGB]{7,59,76}0.68 & 0.69 & \color[RGB]{7,59,76}0.01 \end{array} } \]

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{`call'} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{`poll'} & X_1^* & Y_1^* & X_2^* & Y_2^* & \dots & X^*_{625} & Y^*_{625} & \overline{X}^* & \overline{Y}^* &\frac{\sum_{i:X_i^*=0} Y_i}{\sum_{i:X_i^*=0} 1} & \frac{\sum_{i:X_i^*=1} Y_i}{\sum_{i:X_i^*=1} 1} & \text{difference} \\ \hline \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}X_{398} & \color[RGB]{239,71,111}Y_{398} & & & \color[RGB]{239,71,111}\dots & & & & & & & & \\ & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0 & & & \color[RGB]{239,71,111}\dots & & & & & & & & \\ \color[RGB]{17,138,178}2 & \color[RGB]{17,138,178}X_{293} & \color[RGB]{17,138,178}Y_{293} & & & \color[RGB]{17,138,178}\dots & & & & & & & & \\ & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & & & \color[RGB]{17,138,178}\dots & & & & & & & & \\ \color[RGB]{6,214,160}3 & \color[RGB]{6,214,160}X_{281} & \color[RGB]{6,214,160}Y_{281} & & & \color[RGB]{6,214,160}\dots & & & & & & & & \\ & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}0 & & & \color[RGB]{6,214,160}\dots & & & & & & & & \\ \end{array} } \]

A Table of Bootstrap Polls

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{call} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{poll} & X_1 & Y_1 & X_2 & Y_2 & \dots & X_{625} & Y_{625} & \overline{X} & \overline{Y} &\frac{\sum_{i:X_i=0} Y_i}{\sum_{i:X_i=0} 1} & \frac{\sum_{i:X_i=1} Y_i}{\sum_{i:X_i=1} 0} & \text{difference} \\ \hline \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}\dots & \color[RGB]{7,59,76}0 & \color[RGB]{7,59,76}1 & \color[RGB]{7,59,76}0.28 & \color[RGB]{7,59,76}0.68 & \color[RGB]{7,59,76}0.68 & 0.69 & \color[RGB]{7,59,76}0.01 \end{array} } \]

\[ \small{ \begin{array}{r|rr|rr|r|rr|rrrr} \text{`call'} & 1 & & 2 & & \dots & 625 & & & & & & \\ \text{`poll'} & X_1^* & Y_1^* & X_2^* & Y_2^* & \dots & X^*_{625} & Y^*_{625} & \overline{X}^* & \overline{Y}^* &\frac{\sum_{i:X_i^*=0} Y_i}{\sum_{i:X_i^*=0} 1} & \frac{\sum_{i:X_i^*=1} Y_i}{\sum_{i:X_i^*=1} 1} & \text{difference} \\ \hline \color[RGB]{239,71,111}2 & \color[RGB]{239,71,111}X_{398} & \color[RGB]{239,71,111}Y_{398} & \color[RGB]{239,71,111}X_{129} & \color[RGB]{239,71,111}Y_{129} & \color[RGB]{239,71,111}\dots & \color[RGB]{239,71,111}X_{232} & \color[RGB]{239,71,111}Y_{232} & & & & & & \\ & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}\dots & \color[RGB]{239,71,111}0 & \color[RGB]{239,71,111}1 & \color[RGB]{239,71,111}0.29 & \color[RGB]{239,71,111}0.68 & \color[RGB]{239,71,111}0.69 & \color[RGB]{239,71,111}0.68 & \color[RGB]{239,71,111}-0.01 \\ \color[RGB]{17,138,178}2 & \color[RGB]{17,138,178}X_{293} & \color[RGB]{17,138,178}Y_{293} & \color[RGB]{17,138,178}X_{526} & \color[RGB]{17,138,178}Y_{526} & \color[RGB]{17,138,178}\dots & \color[RGB]{17,138,178}X_{578} & \color[RGB]{17,138,178}Y_{578} & & & & & & \\ & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}\dots & \color[RGB]{17,138,178}0 & \color[RGB]{17,138,178}1 & \color[RGB]{17,138,178}0.28 & \color[RGB]{17,138,178}0.65 & \color[RGB]{17,138,178}0.64 & \color[RGB]{17,138,178}0.67 & \color[RGB]{17,138,178}0.03 \\ \color[RGB]{6,214,160}1M & \color[RGB]{6,214,160}X_{281} & \color[RGB]{6,214,160}Y_{281} & \color[RGB]{6,214,160}X_{520} & \color[RGB]{6,214,160}Y_{520} & \color[RGB]{6,214,160}\dots & \color[RGB]{6,214,160}X_{363} & \color[RGB]{6,214,160}Y_{363} & & & & & & \\ & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}\dots & \color[RGB]{6,214,160}0 & \color[RGB]{6,214,160}1 & \color[RGB]{6,214,160}0.28 & \color[RGB]{6,214,160}0.68 & \color[RGB]{6,214,160}0.66 & \color[RGB]{6,214,160}0.71 & \color[RGB]{6,214,160}0.05 \\ \end{array} } \]

The Difference’s Bootstrap Sampling Distribution

difference.bootstrap.samples = array(dim=10000)
for(rr in 1:10000) {
    I = sample(1:n, n, replace=TRUE)
    Xstar = X[I]
    Ystar = Y[I]
    difference.bootstrap.samples[rr] = mean(Ystar[Xstar==1]) - mean(Ystar[Xstar==0])
}

• Not bad. Our bootstrap sampling distribution looks a lot like our actual sampling distribution.
• And, in particular, the intervals widths we calculate are very close: 0.156 vs. 0.163.
• And that’s really all that matters. Q. Why?